A Discriminative Model for Learning Semantic and Geometric Interactions in Indoor Scenes∗

Visual scene understanding is a difficult problem, interleaving object detection, geometric reasoning and scene classification. Consider the scene in Fig. 1.(a). A scene classifier will tell you, with some uncertainty, that this is a dining room [6, 3]. A layout estimator [5, 7] will tell you, with different uncertainty, how to fit a box to the room. An object detector [8, 4] will tell you, with large uncertainty, that there is a dining table and four chairs. Each algorithm provides important but uncertain and incomplete piece of information. This is because the scene is cluttered with objects which tend to occlude each other: the dining table occludes the chairs, the chairs occlude the dining table; all of these occlude the room layout components (i.e. the walls and floor).